1. Field of the Invention
This invention relates to data storage systems having write-once-read-many capabilities for enhanced data integrity.
2. Background Information
A file server is a computer that provides file service relating to the organization of information on storage devices, such as disks. The file server or filer includes a storage operating system that implements a file system to logically organize the information as a hierarchical structure of directories and files on the disks. Each “on-disk” file may be implemented as a set of data structures, e.g., disk blocks, configured to store information. A directory, on the other hand, may be implemented as a specially formatted file in which information about other files and directories are stored.
A filer may be further configured to operate according to a client/server model of information delivery to thereby allow many clients to access files stored on a server, e.g., the filer. In this model, the client may comprise an application, such as a database application, executing on a computer that “connects” to the filer over a direct connection or computer network, such as a point-to-point link, shared local area network (LAN), wide area network (WAN), or virtual private network (VPN) implemented over a public network such as the Internet. Each client may request the services of the file system on the filer by issuing file system protocol messages (in the form of packets) to the filer over the network.
A common type of file system is a “write in-place” file system, an example of which is the conventional Berkeley fast file system. By “file system” it is meant generally a structuring of data and metadata on a storage device, such as disks, which permits reading/writing of data on those disks. In a write in-place file system, the locations of the data structures, such as inodes and data blocks, on disk are typically fixed. An inode is a data structure used to store information, such as metadata, about a file, whereas the data blocks are structures used to store the actual data for the file. The information contained in an inode may include, e.g., ownership of the file, access permission for the file, size of the file, file type and references to locations on disk of the data blocks for the file. The references to the locations of the file data are provided by pointers in the inode, which may further reference indirect blocks that, in turn, reference the data blocks, depending upon the quantity of data in the file. Changes to the inodes and data blocks are made “in-place” in accordance with the write in-place file system. If an update to a file extends the quantity of data for the file, an additional data block is allocated and the appropriate inode is updated to reference that data block.
Another type of file system is a write-anywhere file system that does not over-write data on disks. If a data block on disk is retrieved (read) from disk into memory and “dirtied” with new data; the data block is stored (written) to a new location on disk to thereby optimize write performance. A write-anywhere file system may initially assume an optimal layout such that the data is substantially contiguously arranged on disks. The optimal disk layout results in efficient access operations, particularly for sequential read operations, directed to the disks. A particular example of a write-anywhere file system that is configured to operate on a filer is the Write Anywhere File Layout (WAFL™) file system available from Network Appliance, Inc. of Sunnyvale, Calif. The WAFL file system is implemented within a microkernel as part of the overall protocol stack of the filer and associated disk storage. This microkernel is supplied as part of Network Appliance's Data ONTAP™ software, residing on the filer, that processes file-service requests from network-attached clients.
As used herein, the term “storage operating system” generally refers to the computer-executable code operable on a computer that manages data access and may, in the case of a filer, implement file system semantics, such as the Data ONTAP™ storage operating system, implemented as a microkernel, and available from Network Appliance, Inc. of Sunnyvale, Calif., which implements a Write Anywhere File Layout (WAFL™) file system. The storage operating system can also be implemented as an application program operating over a general-purpose operating system, such as UNIX® or Windows NT®, or as a general-purpose operating system with configurable functionality, which is configured for storage applications as described herein.
Disk storage is typically implemented as one or more storage “volumes” that comprise physical storage disks, defining an overall logical arrangement of storage space. Currently available filer implementations can serve a large number of discrete volumes (150 or more, for example). Each volume is associated with its own file system and, for purposes hereof; volume and file system shall generally be used synonymously. The disks within a volume are typically organized as one or more groups of Redundant Array of Independent (or Inexpensive) Disks (RAID). RAID implementations enhance the reliability/integrity of data storage through the redundant writing of data “stripes” across a given number of physical disks in the RAID group, and the appropriate caching of parity information with respect to the striped data. In the example of a WAFL file system, a RAID 4 implementation is advantageously employed. This implementation specifically entails the striping of data across a group of disks, and separate parity caching within a selected disk of the RAID group. As described herein, a volume typically comprises at least one data disk and one associated parity disk (or possibly data/parity partitions in a single disk) arranged according to a RAID 4, or equivalent high-reliability, implementation.
Data storage is an increasingly crucial and central part of many industries dealing in financial transactions and other sensitive tasks, such as banks, government facilities/contractors, defense, health care institutions, pharmaceutical companies and securities brokerages. In many of these environments, it is necessary to store selected data in an immutable and unalterable manner. This need continues to grow in the light of current concerns over institutional fraud and mismanagement, wherein the temptation on the part of wrongdoers to erase or alter incriminating data is always present. Forms of data that require immutable treatment often include e-mails, financial documents and transaction records, and any other record that may act as proof of an important action or decision. Even in less-critical/unregulated environments, the ability to store a secure unalterable data cache is highly desirable. For example engineering, medical, law and other professional firms may wish to establish a cache of key data (e.g. invention reports or design files, client communications, medical images, etc.), that will remain unaltered and online for long periods on time. These caches can provide reliable references and proofs for clients and other interested parties.
For an example of a highly regulated environment, the United States Securities and Exchange Commission (SEC)—the body that regulates all securities transactions and reporting relative to public corporations—promulgates SEC Rule 17a-4 governing document retention for brokers and investment institutions. This rule requires that these entities store e-mails and other documents in connection with a variety of transactions and trades by clients of the entities unchanged and unchangeable for a number of years and to be able to provide these records to the SEC and other regulators on short notice. Failure to comply with these rules can lead to significant sanctions.
The simplest approach to providing an immutable record of selected data is to print out the subject data into hardcopy form and then to store it physically as paper records or microfiche copies according to an established procedure. The drawbacks to this approach are many. The printouts and/or transfer to microfiche must occur regularly and continuously, resulting in a potentially huge investment of worker-hours. Large spaces for physical storage are required. In many instances, the amount of incoming data could simply overwhelm a physical storage system's ability to handle it. Retrieval of such data is inherently slow and human error resulting in the misfiling/non-filing of at least some data is likely. Thus, this approach is unsuited to the ever-growing volume of data requiring immutable storage. A significantly better approach involves the use of non-physical storage media (e.g. optical, tape or disk-based storage).
A common approach to creating a cache of immutable data in a computer-based storage system is to establish a bank of write-once-read-many (also termed “WORM”) storage devices in communication with one or more file servers that handle the data. Basic WORM devices consist of removable tape drives that back up and store the contents of a file server memory/disks at predetermined times. Inherent in the tape drive's functionality is the ability to create a WORM copy of certain stored data. When an administrator directs the production of a WORM tape copy, the drive then records one or more indexed tags on the segment of the tape containing the WORM data. This tag is hence-forth recognized by the drive as a WORM copy and, in response to the tag, the drive prohibits the overwrite of that segment. This approach is effective but still fraught with disadvantages. Tapes have relatively limited storage and must be physically removed and stored when full. This requires human intervention that can lead to loss or misfiling of the tapes and that slows retrieval of the information as the physical tapes must be found, mounted and replayed. In addition, the WORM capability is largely dependent of the drive's recognition of the tags and ability to prevent overwrite. Drives may become broken or obsolete over time and the associated WORM tags may be meaningless to newer drives.
More recent WORM systems use electro-optical drives and an internal stack of optical storage platters (similar to recordable compact disks). These drives are highly effective in creating WORM copies as the platters are chemically etched by a drive laser to generate data patterns, and they can only be etched once in a given storage location. The disadvantage to optical storage is that these drives are expensive in comparison to conventional disks and tapes, and more importantly, still retain significant physical limitations in maximum storage size. In other words, while these drives may be able to easily handle several hundred gigabytes of data, the trend is toward the storage of terabytes of data. As such new drives and platters must be added to the system continuously at a significant cost of materials and labor. Similarly, like tape and other non-disk media, write time may be somewhat slow when compared to conventional disk storage.
An inherently desirable approach to WORM storage would be to employ conventional fault-tolerant (e.g. RAID-based) disk storage (or similar rewritable media) as a platform for a WORM storage system. Such disks are relatively inexpensive and easily added to an existing storage system. However, disks are inherently rewritable and/or erasable, and existing operating systems and protocols are designed with semantics that specifically enable the free rewriting and erasure of attached disks. Any solution that utilizes disks to implement WORM storage must absolutely prevent alteration of WORM-designated data. In addition, to maintain longevity of the solution and make it available to as many clients as possible, the WORM implementation should utilize open protocols such as CIFS and NFS and require minimal alteration to these protocols or the applications that employ them.